MEMORANDUM July 19, 2017 


TO: Gloria Cavazos 
Chief Human Resources Officer 


Grenita Lathan 
Chief Academic Officer 


FROM: Carla Stevens 
Assistant Superintendent, Research and Accountability 


SUBJECT: TEACHER APPRAISAL AND DEVELOPMENT SYSTEM END OF YEAR 
REPORT, 2015-2016 


The Houston Independent School District (HISD) Teacher Appraisal and Development System 
(TADS), launched in 2011-2012, is intended to improve teacher performance and increase 
student achievement. This report documents teacher appraisal outcomes from the 2015-2016 
school year and includes historical data since 2011-2012. Teacher appraisal outcomes are 
presented by summative ratings across multiple years and by specific teacher and campus level 
variables. TADS outcomes are then summarized by each appraisal component — Instructional 
Practice (IP), Professional Expectations (PE), and Student Performance (SP). Finally, this report 
examines the impact of Student Performance on summative ratings. 


Key findings include: 

e In 2015-2016, a total 12,255 teachers worked in HISD and were eligible for appraisal under 
the Teacher Appraisal and Development system. Of these teachers, 11,015 (90%) received 
an overall TADS summative rating. 

e In 2015-2016, 88 percent of teachers were rated as either Effective or Highly Effective in 
their overall summative appraisal rating, the highest proportion since TADS was introduced 
in 2011-2012. 

e The greatest variation in the proportion of appraisal component ratings between years 
occurred within Student Performance. With the change in SP measures used in 2015-2016, 
the proportion of teachers with an SP rating of Level 4 nearly doubled from 35 percent in 
2014-2015 to 67 percent in 2015-2016. 

e In 2015-2016, the proportion of teachers with a Highly Effective summative rating that 
included an SP rating (33%) was 11 percentage points higher than for teachers with a 
Highly Effective summative rating that did not include an SP rating (22%). 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 
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Teacher Appraisal and Development System: 
End of Year Report, 2015-2016 


Executive Summary 


Evaluation Description 

The Houston Independent School District (HISD) launched the Effective Teachers Initiative (ETI) in 2010 
with the goal of increasing teacher effectiveness to improve student achievement. As part of this initiative, 
HISD implemented the Teacher Appraisal and Development System (TADS) in the 2011-2012 school year 
to provide teachers, principals, and district officials the information they need to improve teacher 
performance in the classroom and, thereby, increase student outcomes. 


This report documents teacher appraisal outcomes from the 2015-2016 school year and includes historical 
data since 2011-2012. Teacher appraisal outcomes are presented by summative ratings across multiple 
years, and by campus level variables and teacher characteristics. Teacher appraisal outcomes are then 
described by each TADS component -— Instructional Practice (IP), Professional Expectations (PE), and 
Student Performance (SP). Finally, this report examines the impact of Student Performance on summative 
ratings. 


Highlights 

e In2015-2016, 90 percent of teachers (11,015 of 12,255) teachers received an overall TADS summative 
rating. Of those teachers, 88 percent were rated as either Effective or Highly Effective in their overall 
summative appraisal rating, the highest proportion since TADS was introduced in 2011-2012. From 
2012-2013 to 2015-2016, the mean summative score for all appraised teachers increased from 2.93 
to 3.16. 


e Met Standard schools had more than double the proportion of Highly Effective teachers (27%) 
compared to teachers at IR schools (12%). At the same time, teachers in IR schools (23%) were more 
than twice as likely to be rated as Needs Improvement compared to teachers in Met Standard schools 
(10%). 


e New teachers, with less than one year of experience, were over three times more likely to receive a 
summative rating of Needs Improvement or Ineffective compared to their more experienced colleagues 
(34% compared to 10% for all other teachers). 


e Teachers were over four times more likely to receive a Level 1 or Level 2 rating for Instructional Practice 
(IP) than for Professional Expectations (PE) (13% compared to 3%). 


e =In 2015-2016, by board decision, the district excluded Value-Added and normative assessments from 
the SP calculations. Consequently, SP ratings for 2015-2016 included only measures of Comparative 
Growth for TELPAS and Student Progress. With the change in SP measures, the proportion of teachers 
with an SP rating included in their summative rating fell thirteen percentage points from 2014-2015 to 
2015-2016 (43% to 30%). In 2015-2016, teachers were twice as likely to receive a rating of Level 4 
for Student Performance (SP) (67%) compared to IP (24% of teachers were rated Level 4) and PE 
(29% of teachers were rated Level 4). 
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The proportion of teachers with a Highly Effective summative rating that included an SP rating (83%) 
was 11 percentage points higher than the proportion of teachers with a Highly Effective summative 
rating that did not include an SP rating (22%). At the same time, the proportion of teachers with a Needs 
Improvement summative rating that included an SP rating (6%) was eight percentage points lower than 
the proportion of teachers with a Needs Improvement summative rating that did not include an SP rating 
(14%). 


Recommendations 


HISD’s teacher appraisal system was designed to provide teachers with consistent, individualized 
support in order to have the greatest impact on student learning outcomes in the classroom. HISD 
should consider studying possible correlations between TADS and the facilitation of instructional 
support to improve teacher effectiveness, possibly by analyzing longitudinal and qualitative data from 
teachers and appraisers that have participated in TADS across multiple years. 


In efforts to ensure fair support for all teachers, the district should continue to encourage greater TADS 
participation. One potential area for expanding teacher participation is to include teachers assigned to 
district charter schools. 


Teachers may not have adequate support to overcome the obstacles and challenges they face at low- 
performing schools. HISD should continue to focus on growing and supporting teachers located at 
Improvement Required (IR) schools who typically receive lower ratings than teachers at Met Standard 
schools. In addition, HISD should continue efforts that attract effective teachers to IR schools. Future 
research could explore the possible reasons why teachers at IR schools receive lower ratings through 
analysis of longitudinal and qualitative data for teachers and appraisers that have participated in TADS 
across multiple years. 


Teachers with less than five years of experience require individualized support focused on improving 
their effectiveness in the classroom. HISD should continue to support ongoing strategies for these 
teachers, including efforts to retain effective teachers beyond their first few years of teaching. 


The types of measurements used to calculate the Student Performance component of the TADS system 
have varied across years. Consequently, two areas of TADS have been impacted by these changes. 
First, the proportion of teachers that have SP available for their summative rating substantially 
decreased in 2015-2016. Second, the number of statistically rigorous, objective measures used to 
calculate SP has decreased, resulting in Student Progress, a subjective, less-rigorous measure, 
becoming the predominate measure for SP. TADS leadership should continue ongoing efforts to ensure 
the accuracy and consistency of all student growth measures used to calculate the SP rating (i.e., 
Value-Added growth, Comparative Growth, and Student Progress). 


Findings from this report suggest that the Student Progress measure may increase a teacher’s SP 
score and, consequently, increase overall summative appraisal ratings for teachers that use Student 
Progress as a measure. Ensuring the accuracy, validity, and stability of the Student Progress measure 
is an ongoing effort and should remain a focus of TADS leadership. 
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Introduction 


Houston Independent School District's Teacher Appraisal and Development System (TADS), as part of the 
Effective Teachers Initiative (ETI), is designed to promote effective teaching in every classroom, which 
ultimately translates into improved student achievement. TADS is intended to give teachers, principals, and 
district officials the information they need to improve teacher performance in the classroom that ensures 
the opportunity for every student in the district to learn from an effective teacher (Research and 
Accountability, 2015). TADS, similar to other well-designed evaluation systems, incorporates multiple, 
differently-weighted measures of classroom observations and student growth to evaluate teacher 
effectiveness. A 2016 external audit reported that TADS is statistically comparable to other comprehensive 
evaluation systems in large districts with available data (Education Analytics, 2016). 


In the TADS system, effective teaching is determined by three performance criteria areas, or appraisal 
components — Instructional Practice (IP), Professional Expectations (PE), and Student Performance (SP). 
Each appraisal component is based on specific criteria. Further information on the TADS component 
distribution can be found in Appendix A (p.33). Teachers are rated on a scale of one to four for each of the 
appraisal components. The weighted sum of those appraisal components is then used to calculate a 
teacher’s TADS summative appraisal rating. The components used to calculate a teacher's summative 
rating vary depending on the measures available to a teacher. Teachers must have at least two measures 
of student growth or achievement to have SP count in their summative ratings. If a teacher has only one 
SP measure or no SP measure, the overall TADS summative rating is calculated using 70 percent IP and 
30 percent PE ratings. Teachers that receive all three appraisal components (i.e., IP, PE, and SP) receive 
a summative rating based on 50 percent IP, 20 percent PE, and 30 percent SP (Figure 1). A detailed guide 
of the summative rating components can be found in Appendix B (p. 34). 


Figure 1. TADS Summative Rating Components Weights, 2015-2016 
Summative Rating without SP Summative Rating with SP 


Instructional 


Instructional 
Practice (IP) Student 


50% Performance 
(SP) 30% 


Practice (IP) 
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Source: Student Performance (SP) Guidebook, 2015-2016 


For the summative appraisal rating, weighted by available appraisal components, each teacher receives a 
rating of Ineffective, Needs Improvement, Effective, or Highly Effective. These ratings are scored as: 1.00 
to 1.49 — Ineffective, 1.50 to 2.49 — Needs Improvement, 2.50 to 3.49 — Effective, and 3.50 to 4.00 — Highly 
Effective. 


Over the course of the school year, the TADS system pairs each participating teacher with one appraiser. 
The role of the appraiser is to coach the teacher towards effective teaching practices. Appraisers should 
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observe teachers to provide useful feedback intended to improve teaching practices and support the 
teacher in curriculum planning and professional development (Leadership Development, 2013). Appraisers 
assign scores to the teachers for whom they are responsible using standardized rubrics and support 
teachers in determining outcome measures. Appraisers use the IP rubric to assess a teacher’s skills and 
ability to promote learning through classroom observations and walkthroughs. Appraisers use the PE rubric 
to assess a teacher’s efforts to meet objective, measurable standards of professionalism. Along with IP and 
PE, appraisers support teachers through the Student Performance (SP) process, including approving a 
teacher’s student outcome measures. Appendix A (p. 33) presents the various types of weights used to 
calculate a teacher's SP rating. Further details for each Student Performance measure can be found in 
Appendix C (pp. 35-36). 


The goal of this report is to describe the distribution of TADS teacher summative ratings and the 
performance area appraisal components used to construct a teacher’s overall summative appraisal rating. 
Summative appraisal ratings are described over the five-year duration of HISD’s appraisal system for 
teachers and by comparisons between the 2014-2015 and 2015-2016 school years. This report examines 
how these ratings were distributed across key variables by campus level and teacher characteristics. 
Teacher appraisal outcomes are then described by each TADS component — Instructional Practice (IP), 
Professional Expectations (PE), and Student Performance (SP). Finally, this report examines the potential 
impact of Student Performance on summative ratings. 


Methods 


Teacher appraisal data were collected from 2011-2012 to 2015-2016. Ratings for Instructional Practice 
(IP), Professional Expectations (PE), Student Performance (SP), and overall summative ratings were 
collected through the TADS Feedback and Development (F&D) Tool and TADS Student Performance (SP) 
Tool. A teacher was eligible for appraisal if s/he was actively employed from the beginning of the school 
year until the end of April of each academic year. Teachers not included in the TADS system may have 
been excluded for a variety of reasons. For example, teachers may not have been rated due to late hiring, 
job title changes, incorrect job titles in HISD Human Resources Information Systems (HRIS), or split roles 
that required teachers to teach students less than 50 percent of the instructional day. Moreover, some 
teachers in leadership roles were appraised in ePerformance, the School Leader Appraisal Tool, rather 
than in TADS. Finally, teachers employed in HISD charter schools were not appraised in TADS. 


For this report, HISD Human Resources (HR) provided district-wide employee rosters, which included 
multiple identifiers for teacher level data. The ASPIRE (Accelerating Student Progress: Increasing Results 
and Expectations) team, housed within the HISD Department of Research and Accountability, provided 
additional identifiers for teacher level data. In each case, only teachers who received a TADS summative 
rating were included in the analyses. Additional data on school accountability were obtained through the 
Texas Education Agency (TEA) website. The specific methodology on developing the specific variables 
used in this report can be found in Appendix D (pp. 37-38). 


Data Limitations 

Changes to the measurements included to calculate the Student Performance TADS component pose a 
challenge to data analysis and comparisons of TADS ratings over time. For 2015-2016, these changes 
refer to exclusion of norm-referenced assessments (lowa/Logramos) from the SP measure of 
Comparative Growth and the exclusion of Value-Added Growth as an SP measure. These two 
changes had a substantial impact on the comparability with results from prior years. 
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Data limitations specific to this report include smaller samples of appraised teachers when described by 
campus and teacher characteristics. Where indicated, the reader will find footnotes explaining data 
limitations. 


Results 
Summative Ratings 


What was the distribution of summative ratings for teachers districtwide in 2015-2016 compared to 
previous years? 


e Figure 2 compares the summative rating distributions from the inception of the TADS system in 2011- 
2012 through 2015-2016.' More details on the summative rating distribution from 2011-2012 to 2015- 
2016 can be found in Appendix E (Table E-1, pp. 39-40). The TADS summative rating distribution 
varied across years. In 2012-2013, the second year of implementation, teachers received the lowest 
proportion of Effective and Highly Effective summative ratings, when it fell nine percentage points (87% 
in 2011-2012 to 78% in 2012-2013). Over the following three years, from 2013-2014 to 2015-2016, 
the proportion of teachers with Effective and Highly Effective ratings increased each year to its highest 
percentage in 2015-2016 (88%). 


Figure 2. Summative Rating Distribution 2011-2012 through 2015-2016 
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2015-2016 


Note: Percentages may not total 100 due to rounding. 


e The proportions of teachers with summative ratings of Needs Improvement and Highly Effective have 
the greatest variation in the distribution from year to year. In 2011-2012, 12 percent of teachers 
received a Needs Improvement rating. That proportion increased by seven percentage points in 2012- 
2013 (19%) and then steadily decreased over time to 12 percent again in 2015-2016. In 2011-2012, 
26 percent of teachers received a Highly Effective rating. That proportion decreased by seven 


1 In the years preceding 2011-2012, HISD used the state’s Professional Development and Appraisal System (PDAS) 
to appraise its teachers. 
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percentage points in 2012-2013 (19%) and then increased over time to 25 percent in 2015-2016 
(Figure 2). 


Figure 3. Mean Summative Score, 2012-2013 through 
2015-2016 
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Source: Teacher Appraisal and Development F&D Tool, 2012-2013, 
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e Figure 3 shows the mean summative scores from 2012-2013 to 2015-2016. The mean summative 
score for teachers has steadily increased by 0.23 points from 2.93 in 2012-2013 to 3.16 in 2015-2016. 


What was the distribution of summative ratings and appraisal components for teachers districtwide 
in 2015-2016 compared to 2014-2015? 


e =In 2015-2016, a total 12,255 teachers worked in HISD and were eligible for appraisal under TADS. Of 
these teachers, 11,015 (90%) received an overall TADS summative rating compared to 10,847 of 
11,592 (94%) in the previous 2014—2015 school year. 


e Figure 4 (p. 6) shows the distribution of Ineffective, Needs Improvement, Effective, and Highly Effective 
teachers for 2014-2015 and 2015-2016 by summative rating and appraisal components of Instructional 
Practice (IP), Professional Expectations (PE), and Student Performance (SP). Of the appraisal 
components comprising the overall summative rating, both IP and PE remained relatively consistent 
between years, with marginal increases of two percentage points for teachers receiving Level 4 (i.e., 
Highly Effective) IP and PE ratings. Conversely, Student Performance (SP) ratings were more variable. 
More details on the summative rating and appraisal component distribution from 2011-2012 to 2015-— 
2016 can be found in Appendix E (Table E-1, pp. 39-40). 


e The greatest variation in the proportion of appraisal component ratings between years occurred within 
the Student Performance component.? Of the 11,015 teachers receiving a summative rating in 2015— 
2016, 30 percent (n=3,320) received an SP rating, compared to 43 percent (n=4,638) of 10,847 
teachers in 2014-2015. From 2014-2015 to 2015-2016, the proportion of teachers with an SP rating 
of Level 4 (i.e., Highly Effective) nearly doubled from 35 percent in 2014-2015 to 67 percent. The 
proportion of teachers receiving an SP Level 3 (i.e., Effective) rating fell eight percentage points (28% 


2 The calculation of the Student Performance measures was not consistent between 2014-2015 to 2015-2016. The 
SP ratings for 2014-2015 included measures of Valued-Added, Comparative Growth for lowa/Logramos or TELPAS, 
and Student Progress. The SP ratings for 2015-2016 included measures of Comparative Growth for TELPAS and 
Student Progress. 
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to 20%). Finally, the proportion of teachers with an SP Level 1 (i.e., Ineffective) or Level 2 (i.e., Needs 
Improvement) rating decreased 25 percentage points (87% to 12%) (Figure 4). 


Figure 4. TADS Distribution by Summative Rating and by Appraisal Component, 2014-2015 and 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; Teacher Appraisal and 


Development SP Tool, 2014-2015 and 2015-2016 


Note: Percentages may not total 100 due to rounding. *See Data Limitations on p. 4. 


Table 1 (p. 8) shows performance level changes of summative ratings for teachers who received a 
summative rating in both 2014-2015 and 2015-2016. Of the 11,015 teachers that received a 
summative rating for 2015-2016, 78 percent of teachers (n=8,634) also received a summative rating 
for the previous year, 2014-2015. 


The mean summative score for the 8,634 teachers with summative ratings both years increased by 
0.12 points from 2014-2015 to 2015-2016. A paired t-test? was conducted to compare the mean 
summative scores received by teachers that received a summative rating in both 2014-2015 and 
2015-2016. The mean summative score for 2015-2016 was significantly higher (M = 3.23, SD = 0.51) 
compared to 2014-2015 (M= 3.11, SD = 0.50), t (8,633) = 23.37, p<0.01. 


A total of 5,787 teachers rated as Effective in 2014-2015 received a summative rating again in 2015- 
2016. Of those teachers, nearly 18 percent of them (n=1,031) increased their summative rating to 
Highly Effective in 2015-2016 (Table 1, p. 8). 


3 A paired t-test is a statistical procedure to determine whether the difference in means between two groups is significant 
or due to random chance. 
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Table 1. Summative Ratings Changes from 2014-2015 to 2015-2016 for Teachers Who Received a 


Summative Rating in Both Years (N=8,634) 


2014-2015 Summative Ratings 
2015-2016 : Needs Total in 
Summative Ratings | ["effective | yorovement Alec Highly Effective | 95915-2016 
Ineffective 40 
Needs Improvement 675 
Effective 5,418 
Highly Effective 2,501 
Total in 
2014-2015 18 989 5,787 1,840 8,634 


Source: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016 

Notes: Percentages may not total 100 due to rounding. Red represents 1) teachers receiving an Ineffective rating both 
years and 2) teachers who fell to Ineffective rating or Needs Improvement rating in 2015-2016 from a higher rating 
in 2014-2015. Yellow represents teachers 1) remaining as Needs Improvement both years and 2) teachers who fell 
from Highly Effective in 2014-2015 to Effective in 2015— 2016. Green represents 1) teachers who increased their 
ratings from 2014-2015 to 2015-2016 and 2) teachers remaining as Effective or Highly Effective. 


e Of the 989 teachers rated as Needs Improvement in 2014-2015, nearly 69 percent (n=682) increased 
their summative rating to Effective or Highly Effective in 2015-2016 (Table 1). 


e For comparison, Table 2 shows performance level changes of summative ratings for teachers who 
received a summative rating in both 2013-2014 and 2014-2015. Of the 10,847 teachers that received 
a summative rating for 2014-2015, 77 percent of teachers (n=8,313) also received a summative rating 
for the previous year, 2013-2014. 


Table 2. Summative Ratings Changes from 2013-2014 to 2014-2015 for Teachers Who Received a 


Summative Rating in Both Years (N=8,313) 


2013-2014 Summative Ratings 
2014-2015 ‘ Needs : : : Total in 
Summative Ratings INE ieeIs Improvement EVES TEI Suctoats 2014-2015 
Ineffective 31 
Needs Improvement 767 
Effective 5,523 
Highly Effective 1,992 
Total in 
2013-2014 124 1,228 5,051 1,910 8,313 


Source: Teacher Appraisal and Development F&D Tool, 2013-2014 and 2014-2015 

Notes: Percentages may not total 100 due to rounding. Red represents 1) teachers receiving an Ineffective rating both 
years and 2) teachers who fell to Ineffective rating or Needs Improvement rating in 2014-2015 from a higher rating 
in 2013-2014. Yellow represents teachers 1) remaining as Needs Improvement both years and 2) teachers who fell 
from Highly Effective in 2013-2014 to Effective in 2014-2015. Green represents 1) teachers who increased their 
ratings from 2013-2014 to 2014-2015 and 2) teachers remaining as Effective or Highly Effective. 
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e Of5,051 teachers rated as Effective in 2013-2014 who received a summative rating the following year, 
15 percent of them (n=743) increased their summative rating to Highly Effective in 2014-2015 (Table 
2). This was three percentage points lower than the 18 percent of teachers who similarly increased 
their summative rating from 2014-2015 to 2015-2016 (Table 1, p. 8). 


e In2015—2016, the proportion of teachers that fell from a rating of Highly Effective to Effective decreased 
by 15 percentage points compared to the previous years. From 2013-2014 to 2014-2015, 37 percent 
of teachers who received a Highly Effective summative rating in 2013-2014 (n=1,910) fell to an 
Effective summative rating in 2014-2015 (n=700) (Table 2, p. 8). From 2014-2015 to 2015-2016, 22 
percent of teachers who received a Highly Effective summative rating in 2014-2015 (n=1,840) fell to 
an Effective summative rating in 2015-2016 (n=399) (Table 1). 


What was the distribution of summative ratings for teachers by groups in 2015-2016 compared to 
2014-2015? 


School Level 

e Figure 5 shows summative rating distributions by school level — Elementary, Middle, High, and 
Combined School levels. More details on the summative rating distribution by school level can be found 
in Appendix E (Table E-1, pp. 39-40). Consistent with overall summative ratings, the proportion of 
teachers at all school levels rated Highly Effective increased from 2014-2015 to 2015-2016. The 
largest increase in the proportion of teachers with a Highly Effective rating occurred at the Combined 
School level, with an 11 percentage point increase from 2014-2015 to 2015-2016 (32% to 43%). 


Figure 5. Summative Rating Distribution by School Level, 2014-2015 and 2015-2016 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 2014-2015 
and 2015-2016 

Notes: Percentages may not total 100 due to rounding. Teachers located at Community Services, HCC Life Skills, 
EL DAEP, and Beechnut Academy (n=89) are not included. Teachers without HR identifying information (n=35) 
are not included. 
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Consistent with overall summative ratings, the proportion of teachers at most school levels rated 
Effective decreased from 2014-2015 to 2015-2016, with the exception of Middle Schools, where a one 
percentage point increase was observed. The largest decrease in the proportion of teachers with an 
Effective rating was at the Combined School level, with an eleven percentage point decrease from 
2014-2015 to 2015-2016 (58% to 47%) (Figure 5, p. 9). 


In 2015-2016, the distribution of summative ratings for teachers at Combined Level schools was 
inconsistent with the distribution for teachers at other school levels. In 2015-2016, the proportion of 
teachers rated Effective (47%) was at least sixteen percentage points lower than the proportions of 
teachers rated as Effective at other school levels. Moreover, in 2015-2016, the proportion of teachers 
rated Highly Effective (48%) was at least fifteen percentage points higher than the proportions of 
teachers rated Highly Effective at other school levels (Figure 5). 


When separated by school level, there was high variation in the mean summative score across school 
levels. Teachers at the Combined School level received the highest mean summative score, 3.34, and 
teachers at the Middle School level received the lowest mean summative score, 3.06. The mean 
summative score for each school level remained within the range required for an Effective summative 
rating (2.50 to 3.49). More details on the mean summative scores by school level can be found in 
Appendix F (Table F-1, p. 43). 


Accountability Rating 


Figure 6 illustrates the summative ratings distribution by schools’ accountability rating of Improvement 
Required (IR) or Met Standard for 2014-2015 and 2015-2016. More details on the summative rating 
distribution by accountability rating can be found in Appendix E (Table E-1, p. 39-40). Consistent with 
overall summative ratings, both accountability rating groups showed an increase in the proportion of 
teachers rated Highly Effective from 2014-2015 to 2015-2016. 


Figure 6. Summative Rating Distribution by School Accountability Rating, 2014-2015 
and 2015- 2016 
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School Accountability Rating 


Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; TEA Accountability, 
2014-2015 and 2015-2016; HR Roster File, 2014-2015 and 2015-2016 

Note: Percentages may not total 100 due to rounding. Teachers located at Not Rated (NR) schools (n=57) 
are not included. Teachers without HR identifying information (n=35) are not included. 
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e The separation of teachers by their 2015-2016 campus accountability rating showed a gap of 15 
percentage points between the proportion of Highly Effective Teachers at Met Standard and IR schools. 
Specifically, Met Standard schools had more than double the proportion of Highly Effective teachers 
(27%) compared IR schools (12%) (Figure 6, p. 10). 


e The proportion of teachers appraised as Needs Improvement in IR schools was higher than that of 
teachers in Met Standard schools. In 2015-2016, teachers in IR schools (23%) were more than twice 
as likely to be rated as Needs Improvement compared to teachers in Met Standard schools (10%) 
(Figure 6). 


e Both the mean summative score for Met Standard schools, 3.19, and Improvement Required schools, 
2.93, remained within the range required for an Effective summative rating (2.50 to 3.49). The mean 
summative score for teachers at Met Standard schools was significantly higher compared to the mean 
summative score teachers at IR schools. More details on the mean summative scores by accountability 
rating can be found in Appendix F (Table F-2, p. 43). 


Index 1 Score 

e Figure 7 shows summative ratings by school Index 1 scores for 2014-2015 and 2015-2016. Index 1 
scores are an indication of student achievement at the school level, based on student performance on 
the STAAR test. More details on the summative rating distribution by Index 1 score can be found in 
Appendix E (Table E-1, p. 39-40). Consistent with overall summative ratings from 2014-2015 to 2015- 
2016, there was an increase in the proportion of teachers rated as Highly Effective across all groups 
except those with an Index 1 score of 25 or less. 


Figure 7. Summative Rating Distribution by School Index 1 Score Groups, 2014-2015 and 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; TEA Accountability, 2014-2015 
and 2015-2016; HR Roster File, 2014-2015 and 2015-2016 

Notes: Percentages may not total 100 due to rounding. In 2014-2015, there were no schools with an Index 1 score of less 
than 25. In 2015-2016, the schools receiving an Index 1 score of less than 25 (n=38) had an Index 1 Target score of 35. 
Teachers without HR identifying information (n=35) are not included. 
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When separated by 2015-2016 Index 1 scores, there was high variation in the mean summative score 
across Index 1 score groups. The teachers at schools with Index 1 scores less than 25 received the 
lowest mean summative score, 2.91, and the teachers at schools with Index 1 scores greater than 75 
received the highest mean summative score, 3.36. The mean summative scores by Index 1 score 
groups remained within the range required for a summative rating of Effective (2.50 to 3.49). More 
details on the mean summative score by Index 1 score can be found in Appendix F (Table F-3, p. 43). 


Core Foundation 


Figure 8 displays summative ratings by Core Foundation teachers and Non-Core Foundation teachers 
for 2014-2015 and 2015-2016. More details on the summative rating distribution by Core Foundation 
Status can be found in Appendix E (Table E-1, p. 39-40). In 2015-2016, a slightly higher proportion of 
Non-Core Foundation teachers were rated Effective and Highly Effective compared to Core Foundation 
teachers (90% compared to 87%). Accordingly, a slightly lower proportion of Non-Core Foundation 
teachers were rated Needs Improvement and Ineffective compared to Core Foundation teachers (11% 
compared to 13%). 


Figure 8. Summative Rating Distribution by Core Foundation Teacher Status, 
2014-2015 and 2015-2016 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 
2014-2015 and 2015-2016 

Note: Percentages may not total 100 due to rounding. Teachers without HR identifying information 

(n=35) are not included. 


In 2015-2016, the mean summative score for Non-Core Foundation teachers, 3.19, was higher 
compared to the score for Core Foundation teachers, 3.15. The mean summative score by Core 
Foundation status remained within the range required for an Effective summative rating (2.50 to 3.49). 
More details on the mean summative scores by Core Foundation status can be found in Appendix F 
(Table F-4, p. 44). 


Critical Shortage 


Figure 9 (p.13) displays summative ratings by critical shortage and non-critical shortage status for 
2014-2015 and 2015-2016. More details on the summative rating distribution by critical shortage status 
can be found in Appendix E (Table E-1, p.39—40). Critical shortage teachers and non-critical shortage 
teachers had similar proportions of summative rating performance levels. In 2015-2016, 88 percent of 
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critical shortage teachers and 87 percent of non-critical shortage teachers were rated Highly Effective 
or Effective. 


Figure 9. Summative Rating Distribution by Critical Shortage Teacher Status, 
2014-2015 and 2015-2016 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; 
HR Roster File, 2014-2015 and 2015-2016 

Note: Percentages may not total 100 due to rounding. Teachers without HR identifying 

information (n=35) are not included. 


Consistent with overall summative ratings, the proportion of teachers rated Highly Effective for both 
critical shortage and non-critical shortage groups increased in 2015-2016 compared to 2014-2015. 
Moreover, the proportion of teachers rated as Effective or Needs Improvement decreased slightly and 
the proportion of teachers rated as Ineffective remained stable (Figure 9). 


The mean summative score for critical shortage teachers, 3.17, was higher compared to the score for 
non-critical shortage teachers, 3.14. The mean summative scores by critical shortage status remained 
within the range required for an Effective summative rating (2.50 to 3.49). More details on the mean 
summative score by critical shortage status can be found in Appendix F (Table F-5, p. 44). 


Teachers’ Total Years of Experience 


Figure 10 (p.14) illustrates the summative rating distribution by teachers’ total years of experience for 
2014-2015 and 2015-2016. More details on the summative rating distribution by teachers’ total years 
of experience can be found in Appendix E (Table E-1, p. 39-40). In 2015-2016, the majority of teachers 
within each group by total years of experience were rated Effective. Teachers with one to five years of 
experience had the highest proportion of teachers rated Effective (67%). 


Consistent with overall summative ratings, the proportion of teachers at all experience group levels rated 
Highly Effective increased from 2014-2015 to 2015-2016. The greatest increase in the proportion of 
teachers with a Highly Effective rating occurred for teachers with 6 to 10 years and over 20 years of 
experience, with a seven percentage point increase from 2014-2015 to 2015-2016 (23% to 30% and 
26% to 33%, respectively). Teachers with over 20 years of experience had the highest proportion of 
teachers rated as Highly Effective in 2015-2016 (33%) (Figure 10). 
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Figure 10. Summative Rating Distribution by Teacher Total Years of Experience, 2014-2015 and 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 2014-2015 and 
2015-2016 
Note: Percentages may not total 100 due to rounding. Teachers without HR identifying information (n=35) are not included. 


e New teachers, with less than one year of experience, overall were more than three times more likely to 
be rated Ineffective or Needs Improvement compared to their more experienced colleagues (84% 
compared to 10% for all other teachers‘) (Figure 10). The data showing new teachers as having a higher 
proportion rated as Ineffective or Needs Improvement compared to more experienced teachers has held 
true for the last four school years as well. For a comparison of previous years, see Appendix E (Table 
E-1, p. 39-40). 


e Teachers with over 20 years of experience saw the greatest change of summative ratings distributions 
from 2014-2015 to 2015-2016. The proportion of teachers rated as Effective decreased by eight 
percentage points (65% to 57%) and the proportion of teachers rated as Highly Effective increased by 
seven percentage points (26% to 33%) (Figure 10). 


e There was high variation in the mean summative scores across groups of teachers’ total years of 
experience. New teachers, with less than one year of experience, received the lowest mean summative 
score, 2.79. Teachers with 1 to 5 years of experience received a mean summative score of 3.12. 
Teachers with 6 to 10 years, 11 to 20 years, and more than 20 years of experience received similar 
mean summative scores (3.23, 3.24, and 3.26, respectively). The mean summative scores by total years 
of experience remained within the range required for an Effective summative rating (2.50 to 3.49). More 
details on the mean summative score by teachers’ total years of experience can be found in Appendix 
F (Table F-6, p. 44). 


4 Calculated as the percentage of teachers who were not new who received an Ineffective or Needs Improvement 
rating (966 out of 9,807 teachers). 
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Retention 


Figure 11 displays teacher retention by summative ratings for 2014-2015 and 2015-2016. More details 
on the summative rating distribution by teacher retention can be found in Appendix E (Table E-1, p. 39- 
40). Regarding total retention for 2015-2016 to 2016-2017, 87 percent of teachers that received a 
summative rating during the 2015-2016 school year (9,511 of 10,980) remained in HISD. 


The majority of the 2015-2016 teachers retained for the 2016-2017 school year (90%) received an 
Effective or Highly Effective summative rating in 2015-2016. (Figure 11). 


Figure 11. Summative Rating Distribution by Teacher Retention, 2014-2015 and 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 
2014-2015 and 2015-2016 

Note: Percentages may not total 100 due to rounding. In 2014—2015, there were 59 teachers without HR 
identifying information. In 2015-2016, there were 35 teachers without HR identifying information. 


In 2015-2016, over one quarter (28%) of teachers who exited HISD had a Needs Improvement or 
Ineffective summative rating, a decrease of three percentage points from the previous school year. While 
28 percent of exited teachers left with an Ineffective or Needs Improvement rating, the majority of 
teachers that left (72%) had an Effective or Highly Effective rating (Figure 11). 


The mean summative score for teachers that chose to remain with HISD, 3.19, was higher compared to 
teachers that exited the district, 2.90. The mean summative scores by retention status remained within 
the range required for an Effective summative rating (2.50 to 3.49). More details on the mean summative 
rating by retention can be found in Appendix F (Table F-7, p. 45). 


Teacher Movement 


Figure 12 (p.16) describes teacher movement by summative ratings. More details on the summative 
rating distribution by teacher movement can be found in Appendix E (Table E-1, p. 38-39). Regarding 
teacher movement, 92 percent of teachers (n=8,763) who received a summative rating during the 2015-— 
2016 school year and remained in HISD (n=9,511) did not change locations within HISD for the following 
2016-2017 school year. 
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Figure 12. Summative Rating Distribution by Teacher Movement, 2014-2015 to 2015-2016 
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Note: Percentages may not total 100 due to rounding. 


Overall, the proportion of teachers who moved to a new location received lower summative ratings than 
teachers that remained at their 2014-2015 school. In 2015-2016, teachers who moved to a new location 
had a higher proportion of Ineffective and Needs Improvement ratings than teachers who remained at 
the same school (19% compared to 10%). The proportion of teachers rated Highly Effective who had 
moved to a new location (17%) was 10 percentage points lower than for teachers that remained at their 
same school (27%). However, the proportion of teachers rated Effective was equal for both groups in 
2015-2016 (Figure 12). 


The mean 2015-2016 summative score for teachers that remained at the same school, 3.21, was higher 
compared to the mean score of teachers that moved to a new location, 3.03. The mean summative 
scores by teacher movement status remained within the range required for an Effective summative rating 
(2.50 to 3.49). More details on the mean summative score by teacher movement can be found in 
Appendix F (Table F-8, p. 45) 


Instructional Practice Ratings 


What was the distribution of Instructional Practice (IP) ratings in 2015-2016 compared to previous 
years? 


Figure 13 (p.17) shows the IP ratings distribution from 2011-2012 through 2015-2016. Instructional 
Practice ratings have remained relatively stable. More details of the IP rating distribution from 2011- 
2012 to 2015-2016 can be found in Appendix E (Table E-2, p. 41). Across the five school years, the 
majority of teachers received a Level 3 IP rating. From 2011-2012 to 2015-2016, the proportion of 
teachers earning a Level 3 IP rating decreased seven percentage points. In 2015-2016, 63 percent of 
teachers were rated Level 3, a one percentage point decrease from 2014-2015. 
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Figure 13. Instructional Practice (IP) Rating Distribution 2011-2012 through 2015-2016 
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e The proportion of teachers earning a Level 4 IP rating increased seven percentage points from 201 1- 
2012 to 2015-2016. From 2014—2015 to 2015-2016, there was an increase of two percentage points 
in the proportion of teachers rated Level 4, from 22 percent to 24 percent. This proportion has increased 
steadily since 2011-2012 and 2012-2013, when it was 17 percent (Figure 13). 


e Table 3 shows the IP rating changes for teachers who received a IP rating in both 2014-2015 and 
2015-2016. Of the 11,015 teachers that received an IP rating for 2015-2016, 78 percent (n=8,634) of 
teachers received an IP rating for both 2014-2015 and 2015-2016. 


Table 3. Instructional Practice (IP) Changes for Teachers with Consecutive IP Ratings, 2014-2015 


through 2015-2016 (N=8,634) 
2014-2015 Summative Ratings 


2015-2016 Total in 
Summative Ratings IP Level 1 IP Level 2 IP Level 3 IP Level 4 2015-2016 
IP Level 1 69 
IP Level 2 714 
IP Level 3 5,497 
IP Level 4 2,354 
Total in 
2014-2015 nH) 943 5,708 1,940 8,634 


Source: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016 

Notes: Percentages may not total 100 due to rounding. Red represents 1) teachers receiving an IP Level 1 rating both 
years and 2) teachers who fell to IP Level 1 or IP Level 2 in 2015-2016 from a higher rating in 2014-2015. Yellow 
represents teachers 1) remaining as IP Level 2 both years and 2) teachers who fell from IP Level 4 in 2014-2015 to 
IP Level 3 in 2015-2016. Green represents 1) teachers who increased their ratings from 2014-2015 to 2015-2016 
and 2) teachers remaining as IP Level 3 or IP Level 4. 
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e =A total of 5,708 teachers rated as IP Level 3 in 2014-2015 received an IP rating again in 2015-2016. 
Of those teachers, nearly 15 percent of them (n=845) increased their IP rating to Level 4 and 78 
percent (n=4,465) maintained a Level 3 rating in 2015-2016 (Table 3, p. 17). 


e Of the 943 teachers rated as Needs Improvement in 2014—2015, 63 percent (n=598) increased their IP 
rating to Level 3 or Level 4 in 2015-2016 (Table 3). 


e Apaired t-test> was conducted to compare the mean IP score received by teachers that received an IP 
rating in both 2014-2015 and 2015-2016. The mean IP score for the 8,634 teachers appraised both 
years increased by 0.06 points from 2014—2015 to 2015-2016. The mean IP score for 2015-2016 was 
significantly higher (M = 3.17, SD = 0.61) compared to 2014-2015 (M = 3.11, SD = 0.59), t (8,633) = 
11.74, p<0.01. 


What was the distribution of Instructional Practice (IP) ratings in 2015-2016 compared to 2014-2015 
and by groups? 


School Level 
e Figure 14 shows IP ratings by school level for 2014-2015 and 2015-2016. More details on the IP rating 


distribution by school level can be found in Appendix E (Table E-2, p. 41). The proportion of teachers 
at Elementary, High School, and Combined school levels with an IP Level 4 rating increased from 
2014-2015 to 2015-2016. 


Figure 14. Instructional Practice (IP) Rating Distribution by School Level, 2014-2015 and 2015-2016 
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Source: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 2014-2015 
and 2015-2016 

Note: Percentages may not equal 100 due to rounding. Teachers located at Community Services, HCC Life Skills, 
EL DAEP, and Beechnut Academy (n=89) are not included. Teachers without HR identifying information (n=35) 


5 A paired t-test is a statistical procedure to determine whether the difference in means between two groups is significant 
or due to random chance. 
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In 2015-2016, the distribution of IP ratings for teachers at Combined Level schools was inconsistent 
with the distribution of IP ratings for teachers at other school levels. In 2015-2016, the proportion of 
teachers rated IP Level 3 (47%) was at least sixteen percentage points lower than the proportions of 
teachers rated IP Level 3 at other school levels. At the same time, for 2015-2016, the proportion of 
teachers rated Level 4 (42%) was at least sixteen percentage points higher than the proportions of 
teachers rated IP Level 4 at other school levels (Figure 14, p. 18). 


Accountability Rating 


Figure 15 displays teachers’ IP ratings by school accountability rating for 2014-2015 and 2015-2016. 
More details on the IP rating distribution by accountability rating can be found in Appendix E (Table E- 
2, p. 41). Similar to the summative rating distribution by accountability rating (Figure 6, p. 10), the 
separation of teachers by their campus accountability rating showed a gap between Met Standard and 
IR schools. In 2015-2016, Met Standard schools had more than double the proportion of IP Level 4 
teachers (25%) compared to teachers at IR schools (12%). 


In 2015-2016, the proportion of teachers appraised as Level 2 in IR schools was higher than that of 
teachers in Met Standard schools. Teachers in IR schools (22%) were twice as likely to receive an IP 
Level 2 rating compared to teachers in Met Standard schools (11%) (Figure 15). 


Figure 15. Instructional Practice (IP) Rating Distribution by School Accountability 
Rating, 2014-2015 and 2015-2016 
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Sources: Teacher Appraisal and Development F&D Tool, 2014-2015 and 2015-2016; HR Roster File, 
2014-2015 and 2015-2016; TEA Accountability Ratings, 2014-2015 and 2015-2016 

Note: Percentages may not total 100 due to rounding. Teachers located at Not Rated (NR) schools 
(n=57) are not included. Teachers without HR identifying information (n=35) are not included. 


From 2014-2015 to 2015-2016, the proportion of teachers receiving an IP Level 3 decreased slightly in 
both groups. At IR schools, the proportion of teachers with a Level 3 rating decreased three percentage 
points (67% to 64%), and at Met Standard schools, the proportion of teachers with a Level 3 rating 
decreased one percentage point (64% to 63%) (Figure 15). 
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Index 1 Score 


Figure 16 illustrates teachers’ IP ratings across categories of schools’ Index 1 scores for 2014-2015 
and 2015-2016. More details on the IP rating distribution by Index 1 score can be found in Appendix E 
(Table E-2, p. 41). Similar to the distribution of summative ratings by Index 1 score group (Figure 7, p. 
11), the greatest change in the IP rating distribution by Index 1 score from 2014-2015 to 2015-2016 
was for teachers in schools with an Index 1 score between 26 and 50. 


Figure 16. Instructional Practice (IP) Rating Distribution by Index 1 Score, 2014—2015 and 2015-2016 
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2015-2016; TEA Accountability Ratings, 2014-2015 and 2015-2016 


Note: Percentages may not total 100 due to rounding. In 2014-2015, there were no schools with an Index 1 score of less 


than 25. In 2015-2016, the schools receiving an Index 1 score of less than 25 (n=38) had an Index 1 Target score 
of 35. Teachers without HR identifying information (n=35) are not included. 


The proportion of teachers with an Index 1 score between 26 and 50 rated as Level 4 increased five 
percentage points (6% to 11%) and the proportion of teachers rated as Level 3 decreased two 
percentage points (67% to 65%). At the same time, there was also a decrease in the percentage of 
teachers in schools with an Index 1 score between 26 and 50 rated Level 2 (23% to 21%) and Level 1 
(4% to 3%) (Figure 16). 


Although the majority of teachers at schools with Index 1 scores of 26 to 50 or less were rated Level 3 
or Level 4 in 2015-2016 for Instructional Practice, these teachers were twice as likely to be rated Level 
1 or Level 2 compared to teachers at schools with Index 1 scores greater than 50 (24% compared to 
12%)® (Figure 16). 


Similar to the distribution of summative ratings by Index 1 score group, the greatest proportion of 
teachers with a Level 3 or Level 4 IP rating were located at schools with Index scores of greater than 
75 (94%). Though teachers within this group have a lower proportion of teachers rated as Level 3 (56%) 
compared to other groups of schools by Index 1 score, this group had a higher percentage of teachers 


6 Calculated as the percentage of teachers at schools with index 1 scores greater than 50 who received an Ineffective 
or Needs Improvement rating (1,242 out of 9,944 teachers). 


HISD Research and Accountability 20 


Percentage 


TADS END OF YEAR REPORT, 2015-2016 


with Level 4 ratings compared to other groups of schools by Index 1 score. In 2015-2016, the proportion 
of teachers at schools with Index scores of greater than 75 with rating of Level 4 (38%) surpassed the 
proportion of teachers with a Level 4 IP rating in schools with Index scores of 51 to 75 (16%) by 22 
percentage points (Figure 16, p. 20). 


Teachers’ Total Years of Experience 


Figure 17 shows IP ratings by teachers’ years of experience for 2014-2015 and 2015-2016. More 
details on the IP rating distribution by teachers’ total years of experience can be found in Appendix E 
(Table E-2, p. 41). Similar to the summative rating distribution by teachers’ years of experience (Figure 
10, p.14), the majority of teachers within each group by total years of experience received a Level 3 IP 
rating. Teachers with 1 to 5 years of experience had the highest proportion of teachers rated as Level 3 
(67%) in 2015-2016. 


New teachers, with less than one year of experience, were more than three times more likely to receive 
a Level 1 or Level 2 IP rating compared to their more experienced colleagues (37% compared to 11% 
for all other teachers)’ (Figure 17). 


Consistent with the distribution in summative ratings for teachers in the district from 2014-2015 to 
2015-2016, the proportion of teachers with a Level 4 IP rating increased slightly or remained stable for 
nearly all groups. During the same time, the proportion of teachers receiving an IP rating of Level 3 
decreased slightly or remained stable across all groups (Figure 17). 


Figure 17. Instructional Practice (IP) Rating Distribution by Teachers’ Years of Experience, 
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Note: Percentages may not total 100 due to rounding. Teachers without HR identifying information (n=35) are not included. 


’ Calculated as the percentage of teachers who were not new who received an Ineffective or Needs Improvement rating 
(1,056 out of 9,807 teachers). 
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Teachers with over 20 years of experience saw the greatest change in IP ratings distribution from 2014— 
2015 to 2015-2016. The proportion of teachers with over 20 years of experience with a Level 4 IP rating 
increased four percentage points (27% to 31%) and the proportion of teachers with a Level 3 IP rating 
decreased five percentage points (64% to 59%) (Figure 17, p. 21). 


Professional Expectations Ratings 


What was the distribution of Professional Expectations (PE) ratings in 2015-2016 compared to 
previous years? 


e Figure 18 shows the PE ratings distribution from 2011-2012 through 2015-2016. The percentage of 
teachers with a Level 4 PE rating has steadily increased by ten percentage points, from 19 percent in 
2011-2012 and 2012-2013 to 29 percent in 2015-2016. At the same time, the proportion of teachers 
with a Level 3 PE rating has steadily decreased by ten percentage points, from 78 percent in 2011- 
2012 to 68 percent in 2015-2016. 


Figure 18. Professional Expectation (PE) Rating Distribution 2011-2012 through 2015-2016 


on —@—PE Level 1 —™—PE Level 2 —#—PE Level 3 —®—PE Level 4 
Le) 
%o 
782 77% ° 
75% 74% °, 
70% 68% 
50% 
oO 
oO 
= 27% 29% 
Cc (>) 
S 25% 9% gy et 08 
Oo 
o 
3% 4% 3% Of, 3% 
0% 19 5 <1% 
2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 
(N=10,180) (N=10,362) (N=10,778) (N=10,847) (N=11,015) 
School Year 
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2015-2016 
Note: Percentages may not equal 100 due to rounding 


e In 2015-2016, teachers were over four times more likely to receive a Level 1 or Level 2 rating for 
Instructional Practice than for Professional Expectations (13% compared to 3%) (Figure 13, p. 17, and 
Figure 18). 
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Student Performance Ratings 


What was the distribution of Student Performance (SP) ratings in 2015-2016 compared to previous 
years? 


Figure 19 shows a 13 percentage point decrease in proportion of teachers receiving an SP rating from 
2014-2015 to 2015—2016.® In 2015-2016, 30 percent (n=3,320) of the 11,015 rated teachers received 
an SP rating along with an IP and PE rating to make up their overall summative rating. 


Figure 19. Teachers with a Student Performance Rating 
2012-2013 through 2015-2016 
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Source: Teacher Appraisal and Development F&D Tool, 2012- 
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Figure 20 (p. 24) shows the distribution of SP ratings from 2012-2013 to 2015-2016. More details on 
the SP rating distribution from 2012-2013 to 2015-2016 can be found in Appendix E (Table E-3, p. 
42). The majority of teachers received an SP rating of Level 4 in 2015-2016. From 2014-2015 to 2015- 
2016, there was a 32 percentage point increase in the proportion of teachers with a Level 4 rating (85% 
to 67%). This change coincides with a substantial change in the measures used to calculate SP (see 
Data Limitations, p. 4). 


From 2014-2015 to 2015-2016, there was decrease of eight percentage points in the proportion of 
teachers rated SP Level 3, from 28 percent to 20 percent. This proportion has decreased steadily since 
2012-2013 and 2013-2014, when 39 percent of teachers were rated as Level 3 (Figure 20). 


From 2012-2013 to 2015-2016, there has been variation in the proportion of teachers with a Level 2 
SP rating. The proportion of teachers rated as Level 2 was highest in 2014-2015, at 25 percent. From 
2014-2015 to 2015-2016, there was a 17 percentage point decrease in the proportion of teachers with 
a Level 2 rating (25% to 8%) (Figure 20). 


8 The calculation of the Student Performance measures was not consistent between the years. The SP ratings for 
2014-2015 included measures of Valued-Added, Comparative Growth for lowa/Logramos or TELPAS, and Student 
Progress. The SP ratings for 2015-2016 included measures of Comparative Growth for TELPAS and Student Progress. 
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e From 2012-2013 to 2015-2016, the proportion of teachers with a Level 1 SP rating has decreased 
steadily each year to its lowest percentage in 2015-2016 (4%). From 2014—2015 to 2015-2016, there 
was an eight percentage point decrease in the proportion of teachers with a Level 1 rating (12% to 4%) 
(Figure 20). 


Figure 20. Student Performance (SP) Ratings 2012-2013 through 2015-2016 
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Note: Percentages may not total 100 due to rounding. 


e Table 4 shows the SP rating changes for teachers who received an SP rating in both 2014-2015 and 
2015-2016. Of the 3,320 teachers that received an SP rating for 2015-2016, 46 percent of teachers 
(n=1,541) also received an SP rating for the previous year, 2014-2015. 


Table 4. Student Performance (SP) Changes for Teachers with Consecutive SP Ratings, 2014-2015 


through 2015-2016 (N=1,541) 


2014-2015 SP Ratings 

2015-2016 Total in 
SP Ratings SP Level 1 SP Level 2 SP Level 3 SP Level 4 2015-2016 
SP Level 1 47 
SP Level 2 121 
SP Level 3 323 
SP Level 4 1,050 

Total in 
2014-2015 a 273 425 742 1,541 


Source: Teacher Appraisal and Development SP Tool, 2014-2015 and 2015-2016 

Notes: Percentages may not total 100 due to rounding. Red represents 1) teachers receiving an SP Level 1 rating both 
years and 2) teachers who fell to SP Level 1 or SP Level 2 in 2015-2016 from a higher rating in 2014-2015. Yellow 
represents teachers 1) remaining as SP Level 2 both years and 2) teachers who fell from SP Level 4 in 2014-2015 
to SP Level 3 in 2015-2016. Green represents 1) teachers who increased their ratings from 2014-2015 to 2015— 
2016 and 2) teachers remaining as SP Level 3 or SP Level 4. 
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e A total of 425 teachers who received a Level 3 SP rating in 2014-2015 received an SP rating the next 
consecutive year. Of those teachers, 59 percent (n=250) increased their SP rating to Level 4 in 2015—- 
2016 (Table 4, p. 24). 


e Of the 273 teachers with a Level 2 SP rating in 2014-2015, 82 percent (n=223) increased their SP 
rating to Level 3 or Level 4 in 2015-2016 (Table 4). 


e Of the 742 teachers who received a Level 4 SP rating in 2014-2015 and received an SP rating the next 
consecutive year, 83 percent (n=617) maintained a Level 4 SP rating in 2015-2016 (Table 4). 


e The mean SP score for the 1,541 teachers with an SP rating for both years increased by 0.26 points 
from 2014-2015 to 2015-2016. A paired t-test? was conducted to compare the mean SP score received 
by teachers that received an SP rating in both 2014-2015 and 2015-2016. The mean SP score for 
teachers with an SP rating for both years was significantly higher in 2015-2016 (M = 3.43, SD = 0.77) 
compared to 2014-2015 (M = 3.17, SD = 0.94), f (1,540) = 10.16, p<0.01. 


Student Performance Measures 


Given the change in measures available for SP calculations, how did the mean scores for Student 
Performance measures change in 2015-2016 compared to 2014-2015? 


e Table 5 (p. 26) shows teachers’ mean SP scores and mean summative scores'® by SP measure 
combination received in 2014-2015 and 2015-2016. More details on the weights used in each 
component can be found in Appendix A (p. 33). From 2014-2015 to 2015-2016, the proportion of 
teachers with an SP rating decreased by 13 percentage points (48% to 30%) (Figure 19, p. 23). For 
only teachers that received an SP rating, the overall mean SP score increased by 0.53 points (2.86 to 
3.39) and the mean summative score increased 0.23 points (3.05 to 3.28) from 2014-2015 to 2015— 
2016. The mean summative score for teachers with an SP rating, 3.28, was 0.12 points higher than the 
mean summative score for all teachers, 3.16 (Figure 3, p. 6). 


e In 2015-2016, the mean SP score and the mean summative score varied by the combination of SP 
measures, Comparative Growth for TELPAS and Student Progress. There was a 0.55 point difference 
between the mean SP score for teachers with an SP rating including Comparative Growth, 2.85, and 
the mean SP score for teachers with an SP rating including Student Progress, 3.40. At the same time, 
there was a 0.22 point difference between the mean summative score for teachers with an SP rating 
including Comparative Growth, 3.07, and the mean summative score for teachers with an SP rating 
including Student Progress, 3.29 (Table 5). 


° A paired t-test is a statistical procedure to determine whether the difference in means between two groups is significant 
or due to random chance. 

10 Teachers that receive all three appraisal components (i.e., IP, PE, and SP) received a summative rating based on 
50 percent IP, 20 percent PE, and 30 percent SP. 
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e Between 2014-2015 and 2015-2016, the measures used to calculate Comparative Growth'' changed 
and the proportion of teachers with an SP rating that included Comparative Growth decreased 61 
percentage points (69% to 8%). At the same time, the mean SP score for teachers with an SP rating 
including a Comparative Growth measure increased 0.31 points (2.54 to 2.85) (Table 5). 


Table 5. Mean Scores by Student Performance Measure Combination, 2014-2015 through 2015-2016 


2014-2015 2015-2016 
Student Performance (SP) Number of | Mean SP ae Number of eau ea 
Combination Teachers Score SUTTER Teachers alt SHES 
Score Score Score 
Overall SP 4,638 2.86 3.05 3,320 3.39 3.28 
; : 58% 
: 1 : : 

SP including Value-Added (2,674) 2.53 2.94 
SP including Comparative 69% 8% 
Growin (3,207) 2.54 2.93 (276) 2.85 3.07 
SP including Student 42% 99% 
Progress? (1,936) 3.37 3.26 (3,293) 3.40 3.29 


Source: Teacher Appraisal and Development F&D and SP Tool, 2014-2015 and 2015-2016 

Notes: Teachers needed at least two measures within these components to receive an SP rating. See Appendix A (p. 32) for more 
information. 

' Indicates all SP measure combinations that include a Value-Added measure. Value-Added was not included as an SP measure 
for 2015-2016. 

2 Indicates all SP measure combinations that include a Comparative Growth measure. In 2014-2015, CG included lowa/Logramos 
or TELPAS assessments. In 2015-2016, CG included TELPAS only. 

3 Indicates all SP measure combinations that include a Student Progress measure. 


e From 2014-2015 to 2015-2016, there was a 57 percentage point increase in the proportion of teachers 
with a SP rating that included Student Progress (42% to 99%). The mean SP score for teachers with 
an SP rating that included Student Progress increased slightly by 0.03 points (3.37 to 3.40) (Table 5). 


e In2014—2015, teachers with an SP rating including Value-Added (58%) and teachers with an SP rating 
including Comparative Growth for lowa/Logramos and/or TELPAS) (69%) received similar mean SP 
scores (2.53 and 2.54, respectively) and similar mean summative scores (2.94 and 2.93, respectively). 
(Table 5). 


Student Performance and Summative Ratings 


What was the impact of Student Performance (SP) on summative ratings in 2015-2016? 


e Figure 21 (p. 27) shows the 2015-2016 distribution of all summative ratings along each SP 
performance level, with SP Level 1 indicating teachers with students that did not meet expectations 
and SP Level 4 indicating teachers with students that exceeded expectations according to the selected 
student performance measures in 2015-2016. Overall, the data show that summative ratings and SP 
ratings were generally aligned. In 2015-2016, 95 percent of teachers rated SP Level 3 received a 
summative rating of Effective or Highly Effective. In addition, nearly 100 percent of teachers with a 


"1 For 2014-2015, the Comparative Growth measure included lowa/Logramos or TELPAS assessments. For 2015-- 
2016, the Comparative Growth measure included TELPAS only. 
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Level 4 SP rating received a summative rating of Effective or higher, with the exception of less than 
one percent (n=6) rated as Needs Improvement. 


Of those teachers with an SP Level 1 rating, 76 percent received a summative rating of Needs 
Improvement. However, no teachers with an SP Level 1 rating received a corresponding summative 
rating of Ineffective and 24 percent of them received a summative rating of Effective. Moreover, of 
teachers with an SP Level 2 rating, 85 percent received a summative rating of Effective or Highly 
Effective (Figure 21). 


Figure 21. Summative Ratings by Student Performance (SP) Levels for All Rated Teachers 
and Measures, 2015-2016 
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Sources: Teacher Appraisal and Development F&D and SP Tool, 2014-2015 and 2015-2016 
Note: Percentages may not total 100 due to rounding. 


Figure 22 (p. 28) shows the summative ratings with an SP rating compared with summative ratings 
without an SP rating for 2014-2015 and 2015-2016. The proportion of teachers with SP included in 
their summative rating rated Highly Effective increased by 10 percentage points (23% to 33%) and the 
proportion of teachers rated as Needs Improvement decreased by nine percentage points (15% to 6%). 


In comparison, the proportion of teachers without SP included in their summative rating rated Highly 
Effective increased by only four percentage points (18% to 22%) and the proportion of teachers rated 
as Needs Improvement increased by one percentage point (13% to 14%) (Figure 22). 
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Figure 22. Summative Ratings with an SP Rating Compared with Summative Ratings without 
and SP Rating, 2014-2015 and 2015-2016 
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Source: Teacher Appraisal and Development SP Tool, 2014-2015 and 2015-2016 
Note: Percentages may not total 100 due to rounding. 


e There was a high variation in the summative ratings distributions for teachers with SP compared to 
teachers without SP in 2015-2016. The proportion of teachers with a Highly Effective summative rating 
that included an SP rating (33%) was 11 percentage points higher than for teachers with a Highly 
Effective summative rating that did not include an SP rating (22%). The proportion of teachers with a 
Needs Improvement summative rating that included an SP rating (6%) was eight percentage points 
lower than for teachers with a Needs Improvement summative rating that did not include an SP rating 
(14%). No teachers with an SP rating received an Ineffective summative rating in 2015-2016 (Figure 
22). 


Discussion 


The 2015-2016 school year marked the fifth year of TADS as HISD’s teacher appraisal system. From 
2014-2015 to 2015-2016, the percentage of teachers appraised in the TADS system decreased by four 
percentage points (94% to 90%). Analysis of summative rating trends in 2015-2016 indicates the proportion 
of Effective or Highly Effective-rated teachers was the highest (88%) since TADS was introduced in 2011— 
2012, including a five percentage point increase (20% to 25%) in the proportion of teachers rated as Highly 
Effective. 


When separated by individual TADS components, both Instructional Practice (IP) and Professional 
Expectations (PE) Level 4 ratings, which are scored based on appraisers’ observations, increased from 
2011-2012 to 2015-2016. One explanation for this incremental increase in IP and PE could be that the 
TADS system was proficient in identifying teachers’ areas of instructional growth and facilitating targeted 
support. In other words, quality, individualized feedback from appraisers may be providing teachers with 
information that improves their performance in the classroom. Research also suggests that there may be 
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unintentional reasons for increases to the observational components of teacher evaluation systems. One 
report found that teachers, regardless of true performance, tend to receive high ratings in evaluation 
systems. Known as the Widget Effect, this pattern was attributed to evaluation systems’ lack of 
differentiation for the variations of teacher effectiveness (Weisberg, et al., 2009). Another possibility for 
increases in IP and PE over the five years could be related to the teacher-appraiser relationship (e.g., 
quality of appraiser recommendations for support, appraiser bias towards a highly regarded teacher, etc.) 
(Education Analytics, 2016). HISD might consider studying possible correlations between TADS and the 
facilitation of instructional support to improve teacher effectiveness, possibly by analyzing longitudinal and 
qualitative data for teachers and appraisers that have participated in TADS across multiple years. 


The distribution of Student Performance (SP) ratings were not consistent with IP and PE distribution rating 
trends in 2015-2016. The differences may be, at least partially, attributed to Student Performance 
measures changing from 2014-2015 to 2015-2016. The SP ratings for 2014-2015 included measures of 
Value-Added, Comparative Growth for lowa/Logramos (i.e., normative assessments) or TELPAS, and 
Student Progress. By board decision, the district excluded Value-Added and normative assessments from 
the SP calculations the following year, 2015-2016. As a result, SP ratings for 2015-2016 included only 
measures of Comparative Growth for TELPAS and Student Progress, a subjective, less-rigorous measure. 
In 2015-2016, teachers were twice as likely to receive a rating of Level 4 for SP (67%) compared to IP 
(24% of teachers were rated Level 4) and PE (29% of teachers were rated Level 4). 


From 2014-2015 to 2015-2016, the proportion of teachers with an SP rating included in their summative 
appraisal rating fell 13 percentage points (43% to 30%). At the same time, the proportion of teachers with 
SP included in their summative rating rated as Highly Effective increased by 10 percentage points (23% to 
33%) and the proportion of teachers rated as Needs Improvement decreased by nine percentage points 
(15% to 6%). For those teachers that received an SP rating, the mean SP score increased by 0.53 points 
(2.86 to 3.39) and the mean summative score increased 0.23 points (3.05 to 3.28) from 2014-2015 to 
2015-2016. Changes to the Student Performance measurements from year to year may impact the 
proportion of teachers able to incorporate SP into their summative rating, the score for the SP rating, and 
consequently, the overall summative score. It is recommended that the district consider maintaining the 
same SP measures for at least two consecutive years to assess the impact of SP measure changes on 
teachers’ ability to use SP in their summative rating and SP rating score. 


Student Progress, similar to student learning objectives (SLOs), is a participatory student growth measure 
in which teachers and their appraisers work together to set appropriate goals for students. Recent studies 
call for more rigorous research on the validity of using student learning goals to measure the relationship 
between student achievement and teacher effectiveness (Tyler, 2011; Lacireno-Paquet, Morgan, & Mello, 
2014). Further analysis of the Student Performance component in 2015-2016 indicates variations in the 
mean SP and summative scores by SP measures. Specifically, when a Student Progress measure is 
included in the SP rating, teachers, on average, appear to receive a higher SP rating than when Student 
Progress is not included in SP. From 2014-2015 to 2015-2016, the proportion of teachers with Student 
Progress included in their SP rating increased by 57 percentage points (42% to 99%). For both 2014-2015 
and 2015-2016, the mean SP score and the mean summative score were higher for teachers that included 
Student Progress in their SP rating than for teachers who did not include Student Progress. In 2015-2016, 
specifically, teachers that included Student Progress as an SP measure, on average, received an SP rating 
0.55 points higher than teachers that included Comparative Growth as an SP measure (3.40 compared to 
2.85, respectively). At the same time, there was a 0.23 point difference between the mean summative score 
for teachers with an SP rating including Comparative Growth, 3.07, and the mean summative score for 
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teachers with an SP rating including Student Progress, 3.29. These findings suggest that the Student 
Progress measure may increase a teacher’s SP score, and consequently, the overall summative rating. 


Findings from trends of summative rating distributions by groups on the campus- and teacher-levels offer 
insight for the district on possibilities for targeted professional development or TADS system improvements. 
Since 2012-2013, Met Standard schools have had higher proportions of teachers with a summative rating 
of Highly Effective than IR schools. In 2015-2016, Met Standard schools had more than double the 
proportion of Highly Effective teachers (27%) compared to teachers at IR schools (12%). At the same time, 
teachers in IR schools (23%) were more than twice as likely to be rated as Needs Improvement compared 
to teachers in Met Standard schools (10%). HISD should focus on growing and supporting teachers at 
campus and teacher level groups who have consistently received lower ratings than teachers in comparison 
groups. In addition, HISD should continue efforts that attract effective teachers to IR schools. Future 
research could explore the possible reasons why teachers at IR schools received lower ratings through 
analysis of longitudinal and qualitative data for teachers and appraisers that have participated in TADS 
across multiple years. 


In 2015-2016, the proportion of teachers with one to five years of experience with a summative rating of 
Highly Effective was nine percentage points lower than teachers with six to 10 years of experience. At the 
same time, the proportion of teachers with one to five years of experience with an Effective summative 
rating was five percentage points higher than teachers with six to 10 years of experience. This finding may 
be explained by research from Henry, Bastian, & Fortner (2011), which found that teachers who continue 
teaching after their fifth year are, on average, more effective than teachers who exit after their third or fourth 
year of teaching. The study notes that teachers continue to make substantial gains in effectiveness through 
their third year of teaching when provided with appropriate coaching and professional development (Henry, 
Bastian, & Fortner, 2011). In other words, it is possible that the different proportions of Highly Effective and 
Effective ratings between teachers with less than five years of experience and teachers with six to 10 years 
of experience is because less effective teachers have been leaving the teaching profession before their 
fifth year of teaching. However, support for teachers after their first year of teaching with less than five years 
of experience may lead to considerable gains in effective instructional practice. HISD should continue to 
support ongoing strategies for these teachers, including efforts to retain effective teachers beyond their first 
few years of teaching. 


This report has examined teacher appraisal outcomes for the 2015-2016 school year and previous years. 
Trends observed in appraisal outcomes by campus- and teacher-level groups can guide decision-makers 
in their work toward accurately rating effective teaching, strengthening professional development and 
support, growing teachers’ capacity for effective teaching, and ultimately placing an effective teacher in 
every classroom. 
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APPENDIX A: TADS Components Distribution 


TADS Components Distribution 2015 — 2016 ONLY 


The component weights are applied to derive the Summative Appraisal Rating (IP, PE, and SP combined).' 


Teachers with two TADS components (i.e. no Student Performance rating) have the following weights within teachers’ Summative Appraisal Ratings. 


Instructional Practice Professional Expectations 
(IP) (PE) 


70% 30% 


Teachers wth three TADS components have the following weights within teachers’ Summative Appraisal Ratings. 


Instructional Practice Professional Expectations Student Performance 
(IP) (PE) (SP) 


50% 20% 30% 


The various types of Student Performance’ ? measures have different weights within the Student Performance final rating. 


SP Measur 


Measure C 


tudent 


' All TADS components, including Student Performance (SP) measures of Comparative Growth and Student Progress, use a 4-point scale 
? Teachers must have a minimum of two Student Performance measures to receive a Student Performance rating included in the summative rating 
° On June 9, 2016, the HISD Board voted not to extend the contract with SAS EVAAS®. Teacher-level Value-Added Growth (EVAAS®) for the 2015-2016 TADS is not included in the 


Student Performance or Summative Ratings 
‘CG is a district measure based on TELPAS and/or STAAR assessments in certain grade levels and subjects. 
® Student Progress is a student learning measure that uses two measures of a) district-wide/pre-approved/appraiser-approved assessments, b) district-wide/pre-approved/appraiser- 


approved performance tasksAvork products, or c) student attainment (Pre-K teachers only). 
Last Updated 4/18/2017 


Source: Leadership Development, 2015b 
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HISD Teacher Appraisal and Development System 


Measure 


Summative 
Rating 
Weight 


Criteria for Measurement 


Planning (PL) 


Instruction (I) 


50% 
OR 


70% 


PL-1 Develops student learning goals 


PL-2 Collects, tracks, and uses student data to drive 
instruction; 


PL-3 Designs effective lesson plans, units, and 
assessments 


| -1 Facilitates organized, student-centered, objective-driven 
lessons 


l-2 Checks for student understanding and responds to 
student misunderstanding 


|-3 Differentiates instruction for student needs by employing 
a variety of instructional strategies 


I-4 Engages students in work that develops higher-level 
thinking skills 


1-5 Maximizes instructional time 


1-6 Communicates content and concepts to students 


1-7 Promotes high expectations for students 


1-8 Students actively participating in lesson activities 


1-9 Sets and implements discipline management 
procedures 


1-10 Builds a positive and respectful classroom environment 


Professionalism 
(PR) 


20% 


OR 


30% 


PR-1 Complies with policies and procedures at school 


PR-2 Treats colleagues with respect throughout all aspects 
of work 


PR-3 Complies with teacher attendance policies 


PR-4 Dresses professionally according to school policy 


PR-5 Collaborates with colleagues 


PR-6 Implements school rules 


PR-7 Communicates with parents throughout the year 


PR-8 Seeks feedback in order to improve performance 


PR-9 Participates in professional development and applies 
learning 


Student 
Performance 
(PR) 


30% 


OR 


N/A 


Value-Added not included in 2015 — 2016 summative rating 


Comparative Growth (CG) on TELPAS grades 3-8 


Student Progress 
e On districtwide, pre-approved, or appraiser- 
approved assessments 
e On districtwide, pre-approved, or appraiser- 
approved tasks 
e Student attainment (Pre-K only) 


Source: Leadership Development, 2013; Leadership Development, 2015b 
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APPENDIX C: 2015-2016 Student Performance Measures in Detail 


The Student Performance Rating (SP) is a composite metric used in teachers’ appraisal ratings when 
applicable. Teachers must have at least two of the following measures for SP to be applied to their overall 
summative rating: 


e Comparative Growth on district-wide assessments; 

e Students’ progress on districtwide assessments, pre-approved assessments, or appraiser- 
approved assessments 

e Students’ progress on districtwide, pre-approved, or appraiser-approved performance tasks or 
products 

e Student attainment on districtwide or appraiser-approved assessments. 


SP ratings are on a scale of 1-4. A teacher must have at least two SP measures to receive an SP rating. 
Teachers who do not receive an SP rating will receive a Summative Appraisal Rating based solely on an 
Instructional Practice (IP) rating and a Professional Expectations (PE) rating assigned by the appraiser. 


Measure #1: Value-Added Growth 

Value-Added Growth is a district-rated measure of the extent to which a student’s average growth meets, 
exceeds, or falls short of average growth of students in the district. Value-added analysis uses a student's 
own academic performance across years, grades, and subjects as a basis for determining his/her average 
growth. EVAAS® was used as the value-added growth measure for teachers with available data in the 
Student Performance (SP) rating for TADS from 2011-2012 through 2014-2015. 


On June 9, 2016, the HISD Board voted not to extend the contract with SAS EVAAS®. As a result, teacher- 
level Value-Added Growth for the 2015-2016 Teacher Appraisal and Development System (TADS) is not 
included in the Student Performance or Summative Ratings. If teachers do not have two Student 
Performance measures besides Value-Added, their summative rating will be calculated using only their 
Instructional Practice and Professional Expectations ratings. 


Measure #2: Comparative Growth on district-wide assessments 
Comparative growth may be used to calculate a teacher’s Student Performance rating. Comparative Growth 


(CG) measures the progress of a teacher’s students on a given assessment compared to all other students 
within the same school district who start at the same test-score level. For 2015-2016, CG relies on the use 
of TELPAS assessments in grades 3-8, and is computed using two consecutive years of students’ scores. 
CG scores are placed on a scale of 1-4. For the 2015 -2016 school year, only TELPAS assessments for 
Reading scale scores in grades 3-8 were used to calculate the CG measure. In prior years, CG was 
calculated using norm-referenced data in grades 2-8. 


Measure #3 & #4: Student Progress 
Student Progress may be used to calculate a teacher’s Student Performance rating. Student Progress is a 


type of TADS Student Performance measure that uses summative or cumulative assessments, 
performance tasks, and work products to measure how much content and skill students learned over the 
duration of a course or year, based on where they started the subject or course. 
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#3. Students’ progress on districtwide assessments, pre-approved assessments, or 


appraiser-approved assessments 
Student Progress is a student learning measure that uses summative assessments to measure how much 


content and skill students learned over the duration of a course or year, based on where they started the 
subject or course. Student Progress is an appraiser-approved rating of the extent to which students learned 
an ambitious and feasible amount of content and skills, taking into account students’ starting points. To 
measure Student Progress, teachers must create Goals Worksheets for no more than two of the courses 
they teach and place students into appropriate starting points based on two pieces of evidence, such as 
past grades or past test scores. Once students have been placed into an appropriate starting group, which 
must be approved by the teacher’s appraiser, they will receive a goal dependent upon which assessment 
is appropriate for that course. Assessment results are entered into a Results Worksheet either automatically 
or by the teacher. Once the Results Worksheets have been approved by the appraiser, a teacher will 
receive a Performance Level rating based on how many students achieved their goals. Performance Levels 
are on a scale of 1-4. 


#4. Students’ progress on districtwide, pre-approved, or appraiser-approved performance 
tasks or products 
The Student Progress process using appraiser-approved culminating performance tasks or work products 
mirrors the process for Student Progress on assessments. The only substantive difference is the type of 
summative assessment tool used. For example, in certain subjects, such as art, music, or foreign language, 
a culminating project or performance task might be more appropriate than, or used in conjunction with, a 
more traditional paper-pencil test. 


Measure #5: Student Attainment 
Student Attainment is a student learning measure that uses district-wide or appraiser-approved 


assessments to measure how many students performed at a target level, regardless of their starting point. 
Currently, Student Attainment only applies to Pre-K. 


Source: Leadership Development, 2015b, pp. 35-40 
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APPENDIX D: Methodology for TADS End of Year Report, 2015-2016 


For this report, HISD Human Resources (HR) provided district-wide employee rosters, which included 
multiple identifiers for teacher-level data. The ASPIRE (Accelerating Student Progress: Increasing Results 
and Expectations) team, housed within the HISD Department of Research and Accountability, provided 
additional identifiers for teacher-level data. Additional data on school accountability was obtained through 
the Texas Education Agency (TEA) website. A teacher was eligible for appraisal if s/ne was actively 
employed from the beginning of the school year until the end of April of each academic year. In each case, 
only teachers who received a TADS summative rating were included in the analyses. 


e Teachers were identified using the following criteria: 
o To identify job descriptions specific to teachers, the variable Job Function Code was reported 
as TCH, TEA ELEM, TEA PREK, TEA SEC, or # (i.e., not assigned job function code) 
o To identify salary plans specific to teachers, the variable Salary Plan was reported as RT, VT, 
RO1 or ROS. 


e A teacher’s school level was determined by identifying campus level assignments specific to each 
teacher as indicated in the 2015-2016 HISD District and School Profiles. Teachers located at 
Community Services, HCC Life Skills, EL DAEP, and Beechnut Academy were excluded in this report. 


e Campus accountability ratings and Index 1 Scores were obtained from the TEA using the Texas 
Academic Performance Reports (TAPR) for 2012-2013, 2013-2014, 2014-2015, and 2015-2016. 
TEA did not release accountability data for 2011-2012. 


e Teachers coded by ASPIRE in the student-linkage database as Core Foundation teachers included 
those who taught courses in math, science, social studies, English, and reading. 


e Critical shortage teachers were identified as teaching in a TEA-defined critical shortage or high needs 
area. To be included in this category, the variable Job Family was reported as BIL, MATH, SCIENC, 
SPECIAL ED, ESL, CATE, and/or COMP. 


e Total retention was defined as those teachers from the 2015-2016 school year who remained actively 
employed in HISD in 2016-2017, including those no longer assigned to classrooms. Teachers retained 
in the district were reported by HISD Human Resources Information System (HRIS) as receiving a 
status code of A (active), B (paid leave), C (unpaid leave), F (FMLA Full), or E (FMLA Int). Teachers 
were considered as retained if they were employed from May 2016 through August 2016. 


e Teacher movement was defined as teachers who stayed in the district (those retained) who changed 
locations within HISD from May 2016 to August 2016, regardless of whether the location change 
included a promotion. 


e Teacher years of experience was determined using total professional experience as verified by HRIS. 
Total professional experience is defined as the sum of the variables, HISD Experience and Other 
Experience. Teachers were categorized as new teachers (i.e., in their first year of teaching), 1-5 years, 
6-10 years, 11-20 years, and more than 20 years of experience. 
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e Comparative Growth (CG) is one of five possible measures for Student Performance. CG looks at the 
progress of a teacher’s students compared to all other students within the same school district who 
start at the same level. This measure is calculated by the ASPIRE team, housed within the HISD 
Department of Research and Accountability. For 2015-2016, CG was a district measure based on 
Texas English Language Proficiency Assessment System (TELPAS) assessments for grades 3-8. 
Data were loaded into the TADS F&D Tool for appraisal purposes. 
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Appendix E: Data Tables 


Table E-1. 2011-2012 thro 2015-2016 Summative Rating Distribution by Campus and Teache 


Ineffective (N) Needs Improvement (N) ite MD) Highly Effective (N) BCC IER()) 


2011-  2012—- 2013— -- 2014— -- 2015— | 2011-9 2012— S- 2013— )S- 2014—S- 2015— 2011-9 2012-9 2013-9 2014S 2015-39 2011-9 2012—- 2013- - 2014—- — 2015- 2013- 2014— 2015- 
2012) 2013) 2014S 2015 =. 2016 2014-2015 2016) 2012, 20132014 2015 2016 2012) 2013 2014-2015 = 2016 2015 2016 


School Level 


1% 4% 3% 1% 1% 12% | 23% 18% 15% | 12% | 61% | 57% 58% 65% 63% 26% 17% | 21% | 19% 24% 100% 100% 100% 100% 100% 

Elementary] (54) | (213) | (180) | (54) (42) | (712) | (1,285) | (1,079) | (897) | (737) | (8,473) | (3,250) | (3,441) | (3,862) | (3,808) | (1,460) | (957) | (1,256) | (1,134) | (1,469) | (5,699) (5,705) (5,956) (5,947) (6,056) 
1% 4% 5% 1% 1% 12% | 20% | 20% 18% | 14% | 62% | 54% | 55% 65% 66% 25% | 22% | 20% 16% 18% 100% 100% 100% 100% 100% 

Middle] (24) (67) (91) (14) (18) | (206) | (346) | (361) | (319) | (264) | (1,046) | (937) | (1,005) | (1,182) | (1,209) | (422) | (387) | (859) | (295) | (331) (1,698) (1,737) (1,816) (1,810) (1,822) 
1% 1% 2% 1% 1% 11% | 12% 12% 9% 9% 62% | 69% 67% 67% 63% 26% 19% | 19% | 23% 28% 100% 100% 100% 100% 100% 

High} (22) (23) (42) (16) (15) | (247) | (271) | (280) | (213) | (221) | (1,398) | (1,587) | (1,551) | (1,559) | (1,481) | (586) | (430) | (453) | (524) | (652) (2,253) (2,311) (2,326) (2,312) (2,369) 

2% 2% 2% | <1% | <1% | 12% | 16% 12% 9% 10% | 60% | 58% | 50% 58% 47% 27% | 24% | 37% | 32% 43% 100% 100% 100% 100% 100% 

Combined] (9) (10) (13) (5) (3) (62) (99) (79) (60) (66) | (318) | (351) | (332) | (396) | (325) | (141) | (149) | (246) | (219) | (296) (530) (609) (670) (680) (690) 


Total (N)j 109 313 326 89 78 =| 1,227 | 2,001 | 1,799 | 1,489 | 1,288 | 6,235 | 6,125 | 6,329 | 6,999 | 6,823 | 2,609 | 1,923 | 2,314 | 2,172 | 2,748 10,180 10,362 10,768 10,749? | 10,937 ° 


Accountability Ratingt 


5% 6% 2% 1% 29% | 28% | 27% | 23% 57% 57% 65% 64% 9% 9% 6% 12% 100% 100% 100% 100% 
Improvement Required (IR)} _- (89) | (101) | (37) (21) : (630) | (478) | (621) | (818) : (1,027) | (968) | (1,253) | (900) : (156) | (153) | (121) | (166) : (1,802) (1,700) (1,932) (1,405) 
3% 2% 1% 1% 17% | 15% 11% | 10% 60% 59% 65% 62% 21% | 24% | 23% 27% 100% 100% 100% 100% 
Met Standard] - (224) | (225) | (62) (67) : (1471) | (1,320) |_(967) | (966) : (5,098) | (5,357) | (5,739) | (5,915) : (1,767) | (2,161) | (2,049) | (2,580) : (8,560) (9,063) (8,807) (9,518) 
Total (N)|_- 313 326 89 78 : 2,001 | 1,798 | 1,488 | 1,284 : 6,125 | 6,325 | 6,992 | 6,815 : 1,923 | 2,314 | 2,170 | 2,746 - 10,362 10,763 10,739° | 10,923 ° 
Index 1 Scorest 
5% 25% 0% 0% 24% | 25% 0% 26% 67% 42% 0% 61% 4% 8% 0% 13% 100% 100% 100% 100% 
25 orLess|_ - (3) (3) (0) (0) z (13) (3) (0) (10) z (37) (5) (0) (23) = (2) (1) (0) (5) : (55) (12) (0) (38) 
6% 10% | 3% 1% 32% | 35% | 28% | 21% 53% | 52% 65% 66% 9% 3% 4% 12% 100% 100% 100% 100% 
26 to 50] - (24) (68) (36) (14) : (128) | (246) | (359) | (207) : (214) | (365) | (840) | (661) : (38) (24) (67) (116) : (404) (703) (1,292) (998) 
4% 4% 1% 1% 23% | 21% 16% | 15% 61% 62% 69% 67% 11% | 14% 14% 18% 100% 100% 100% 100% 
51to 75] - (232) | (203) | (60) (64) : (1,247) | (1,134) | (957) | (885) : (3,290) | (3,327) | (4,133) | (4,055) : (597) | (742) | (852) | (1,076) : (5,366) (5,406) (5,992) (6,070) 
1% 1% | <1% | <1% 13% 9% 5% 5% 57% 57% 59% 55% 28% | 33% | 36% 40% 100% 100% 100% 100% 
Greater than 75} - (64) (62) (3) (10) : (609) | (413) | (173) | (186) : (2,566) | (2,615) | (2,062) | (2,123) : (1,285) | (1,547) | (1,266) | (1,555) - (4,514) (4,627) (3,504) (3,874) 
Total (N)|__- 313 326 89 78 : 1,997 | 1,796 | 1,488 | 1,288 : 6,107 | 6,312 | 6,992 | 6,862 : 1,922 | 2,314 | 2,170 | 2,752 : 10,339 10,748 10,788 | 10,9807 
Core Foundation Teachers 
1% 4% 4% 1% 1% | 13% | 22% | 19% 15% | 12% | 60% | 55% 55% 65% 62% 26% | 19% | 22% | 20% 25% 100% 100% 100% 100% 100% 
Core] (75) | (274) | (275) | (68) (87) | (871) | (1,556) | (1,403) | (1,238) | (1,064) | (4,151) | (8,829) | (4,013) | (6,447) | (5,378) | (1,773) | (1,320) | (1,564) | (1,657) | (2,146) | (6,870) (6979) (7,255) (8,410) (8,645) 
1% 1% 1% 1% 1% 11% | 138% | 11% 11% | 10% | 63% | 68% 66% 67% 64% 25% 18% | 21% | 22% 26% 100% 100% 100% 100% 100% 


Non-Core] (34) (39) (51) (21) (21) | (856) | (445) | (396) | (251) | (224) | (2,084) | (2,296) | (2,318) | (1,588) | (1,484) | (836) | (603) | (755) | (518) | (606) (3,310) (3,383) (3,520) (2,378) (2,335) 
Total (N)j|__109 313 326 89 78 | 1,227 | 2,001 | 1,799 | 1,489 | 1,288 | 6,235 | 6,125 | 6,331 | 7,035 | 6,862 | 2,609 | 1,923 | 2,319 | 2,175 | 2,752 10,180 10,362 10,775 10,788 4 10,980 ” 
Source: TADS Feedback and Development Tool; TADS Student Performance Tool; HISD PeopleSoft Rosters: 2011-2012 as of 04-16-2012; 2012-2013 as of 04-10-2013; 2013-2014 as of 04-14-2014; 2014-2015 as of 05-15-2015; 2015-2016 as of 05-28-2016 
tAccountability ratings not available for school year 2011-2012. 
* Retention and Teacher Movement were not calculated in the TADS End of Year Reports for 2011-2012; 2012-2013; and 2013-2014 
‘36 teachers excluded from data. 
? 98 teachers at Community Services, HCC Life Skills, EL DAEP, or with no school identifying information in HR Roster. Not included in school levels. 
3 108 teachers at schools without accountability ratings or no school identifying information in HR Roster. 
“59 teachers without HR Roster identifying information. 
578 teachers at Community Services, HCC Life Skills, EI DAEP, Beechnut Academy, or with no school identifying information in HR Roster. Not included in school levels. 
°57 teachers at schools without accountability ratings or no school identifying information in HR Roster. 
735 teachers without HR Roster identifying information. 
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Table E-1 continued 


Ineffective (N) Needs Improvement (N) Effective (N) Highly Effective (N) SCIEN) 


2012—  2013- 2014- 2012— =2013- = 2014- 2013— =. 2014— 2012- 2013- 2014- 2013- 
2013 2014 =. 2015 2013 2014 =. 2015 2014 2015 2013. 2014 =. 2015 2014 


Critical Shortage Teachers 


100% 100% 100% 100% 100% 

Critical Shortage! (2,389) (2,461) (2,489) (4,081) (4,939) 

100% 100% 100% 100% 100% 

Non-Critical Shortage! (7,791) (7,901) (8,286) (6,707) (6,041 
Total (N) 10,180 10,362 10,775 10,788 * 


Years of Experience 


100% 100% 100% 


100% 


New Teacher| (751) (1,564) | (4,970) | (1,245) | (1,173 
100% 100% 100% 100% 

1-5 Years (3,472) | (2,883) | (3,259) | (3,077) | (3,425 

100% 100% 100% 100% 100% 

6-10 Years (2,419) | (2,277) | (2,107) | (2,188) | (2,116 

100% 100% 100% 100% 100% 

11-20 Years (2,205) | (2,267) | (2,233) | (2,623) | (2,616 

100% 100% 100% 100% 100% 

Over 20 Years (1,333) | (1,255) | (4,202) | (1,655) | (1,650 


Total (N) 10,144' | 10,246 10,771 10,788 * 


Retention* 


100% 


Retained 7 7 = (9,266) (9,511 
100% 100% 
Exited 7 7 = (1,522) (1,469) 


10,788 * 


Teacher Movement* 


100% 
7 - a (8,258) (8,763; 


100% 
Moved to a NewLocation e = 7 (1,008) (748) 
Total (N) = = = 9,266 9,511 

Source: TADS Feedback and Development Tool; TADS Student Performance Tool; HISD PeopleSoft Rosters: 2011-2012 as of 04-16-2012; 2012-2013 as of 04-10-2013; 2013-2014 as of 04-14-2014; 2014-2015 as of 05-15-2015; 2015-2016 as of 05-28-2016 


tAccountability ratings not available for school year 2011-2012. 

* Retention and Teacher Movement were not calculated in the TADS End of Year Reports for 2011-2012; 2012-2013; and 2013-2014 
136 teachers excluded from data. 
?98 teachers at Community Services, HCC Life Skills, EL DAEP, or with no school identifying information in HR Roster. Not included in school levels. 
3 408 teachers at schools without accountability ratings or no school identifying information in HR Roster. 

“59 teachers without HR Roster identifying information. 
5°78 teachers at Community Services, HCC Life Skills, EL DAEP, Beechnut Academy, or with no school identifying information in HR Roster. Not included in school levels. 
°57 teachers at schools without accountability ratings or no school identifying information in HR Roster. 

735 teachers without HR Roster identifying information. 


Remained at the Same School 
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Table E-2. 2011-2012 through 2015—2016 Instructional Practice (IP) Rating Distribution by Campus and Teacher Characteristics 
IP Level 1 (N) IP Level 2 (N) IP Level 3 (N) 


2012— =2013— 2014—- 2015—- 2011— 2012— 2013— 2014— 2015—- 2011— 2012— 2013— 2014— 
2013 2014 2015 2016 2012 2013 2014 + =2015 2016 2012 2013 2014 2015 


TADS END OF YEAR REPORT, 2015-2016 


IP Level 4 (N) 


2012- 
2013 


2013— = 2014— 
2014 2015 


Totals (N) 


2013— 
2014 


Combined 


Total 


Improvement Required (IR) 


100% 
(1,802) 


100% 
(1,700) 


100% 
(1,405) 


100% 
(8,560) 


100% 
(9,063) 


100% 
(9,518) 


10,362 


100% 
(44) 


10,763 


100% 
(6) 


10,923 © 


100% 
(404) 


100% 
(703) 


100% 
(5,366) 


100% 
(5,406) 


100% 
(4,514) 


100% 
(4,627) 


10,328 


10,742 


Critical Shortage 


100% 
(2,389) 


100% 
(2,461) 


100% 
(2,489) 


Non-Critical Shortage, 


100% 
(7,791) 


100% 
(7,901) 


100% 
(8,286) 


10,180 


100% 
(751) 


10,362 


100% 
(1,564) 


10,775 


100% 
(1,970) 


100% 
(1,245) 


100% 
(3,472) 


100% 
(2,883) 


100% 
(3,259) 


100% 
(3,077) 


6-10 Years 


100% 
(2,419) 


100% 
(2,277) 


100% 
(2,233) 


100% 
(2,188) 


11-20 Years 


100% 
(2,205) 


100% 
(2,267) 


100% 
(2,107) 


100% 
(2,623) 


Over 20 Years 


100% 
(1,333) 


100% 
(1,255) 


100% 
(1,202) 


100% 
(1,655) 


Total 


*n<5 

tAccountability ratings not available for school year 2011-2012. 

‘98 teachers at Community Services, HCC Life Skills, EL DAEP, or with no school identifying information in HR Roster were not included in school levels. 

? 408 teachers at schools without accountability ratings or no school identifying information in HR Roster. 

° 59 teachers without HR Roster identifying information. 

*78 teachers at Community Services, HCC Life Skills, EL DAEP, Beechnut Academy, or with no school identifying information in HR Roster. Not included in school levels. 
5 57 teachers at schools without accountability ratings or no school identifying information in HR Roster. 

®° 35 teachers without HR Roster identifying information. 


10,180 


10,246 


Source: TADS Feedback and Development Tool; TADS Student Performance Tool; HISD PeopleSoft Rosters: 2011-2012 as of 04-16-2012; 2012-2013 as of 04-10-2013; 2013-2014 as of 04-14-2014; 2014-2015 as of 05-15-2015; 2015-2016 as of 05-28-2016 


10,771 


10,788 * 
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Table E-3. 2012-2013 through 2015-2016 Student Performance (SP) Ratings by Campus and Teacher Characteristics 
SP Level 1 (N) 


SP Level 2 (N) 


2013— = 2014— = 2015— 2013— = 2014— 
2014 2015 2016 2014 2015 


SP Level 3 (N) 


2013— 
2014 


40% 
(1,006) | (1,157) 


SP Level 4 (N) Totals (N) 


2014— 2015— 740 2013— 2014— 
2015 2016 2014 2015 2014 2015 


100% 


(940) | (439) (2,531) | (3,003) | (3,122) | (2,067) 


38% 40% 


27% 24% 100% 100% 100% 100% 


(327) | (364) | (197) | (123) (851) (916) (721) (514) 
0% | 27% | 16% | 17% 0% 100% 100% 100% 

(0) (7) (78) (97) (0) (23) (473) (557) 

38% | 37% | 24% | 11% 100% | 100% 100% 100% 

Combined (93) | (410) | (77) | (19) (178) 


Total 


Improvement Required (IR) 


1,426 


32% 
(199) 


3,316 


21% 100% 
(137) (179) 


Met Standard 


4% 
(1,227) 


29% 100% 
(1,155) (3,134) 


Total 


25 or Less 


1,426 


1,292 3,313 2 


50% 100% 100% 
(3) (11) (6) 


26 to 50 


52% | 100% 100% 
(88) (112) (170) 


51 to 75 


62% 100% 100% 
(921) | (1,880) (1,481) 


Greater than 75} 


74% 100% 100% 
(1,223) | (1,624) (1,659) 


39% 
(1,426) 


2,235 3,627 3,316 


30% 100% 
(1,227) (2,590) 


0% 
(0) 


14% 100% 
(65) (726) 


Critical Shortage! 


1,426 


39% 
(1,426) 


1,292 3,316 


100% 
(1,470) 


Non-Critical Shortage 


0% 
(0) 


100% 
(1,846) 


Total 


New Teacher] 


1,426 


3,316 


100% 
(260) 


1-5 Years 


100% 
(1,029) 


6-10 Years 


100% 
(665) 


11-20 Years 


100% 
(852) 


Over 20 Years 


100% 
(510) 


Total 


' Six teachers with SP missing from HR Roster file. 
? Three teachers at schools without accountability ratings or no school identifying information in HR Roster. 


Source: TADS Feedback and Development Tool; TADS Student Performance Tool; HISD PeopleSoft Rosters: 2012—2013 as of 04-10-2013; 


3,316 
2013-2014 as of 04-14-2014; 2014-2015 as of 05-15-2015; 2015-2016 as of 05-28-2016 
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Appendix F: Tests of Significance 


Table F-1. One-Way Between Analysis of Variance of Teachers’ Summative Rating by School 


Level Campuses, 2015-2016 


Std. 
Groups N Mean Deviation F p n? 
Elementary 6,056 3.15 0.54 
Middle 1,822 3.06 0.52 
High 2,369 3.20 0.52 
Combined 690 3.34 0.59 
Total 10,937 | 3.15 0.54 F (8, 10,933) = 51.89 | <0.001 | 0.01 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 

Note: Effect size conventions for 7? are: 0.2 = small effect, 0.5 = medium effect, 0.8 = large effect. Teachers located 
at Community Services, HCC Life Skills, EL DAEP, and Beechnut Academy (n=43) were not included. 
Teachers without HR identifying information (n=35) were not included. 


Table F-2. Independent Sample t Test of Teachers Summative Ratings by Accountability Rating 


Campuses, 2015-2016 


Std. 
Groups N Mean Deviation t p g 
Improvement Required 1,405 2.93 0.54 
Met Standard 9,518 3.19 0.53 
Total 10,923 | 3.16 0.54 t(1, 10,921) =-17.12 | <0.001 | -0.49 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 

Note: Effect sizes are calculated for independent sample t-tests using Hedge’s g. Hedge’s g provides a measure of 
effect size weighted to account for different sample sizes. Effect size conventions for Hedge’s g are: 0.2 = 
small effect, 0.5 = medium effect, 0.8 = large effect. Teachers located at Not Rated (NR) schools (n=57) are not 
included in the table. Teachers without HR identifying information (n=35) are not included in the table. 


Table F-3. One-Way Between Analysis of Variance of Teachers’ Summative Rating by Index 1 


psToxe) «= Or- 111) oLUL-\-e-40 oer Is) 


Std. 
Groups N Mean Deviation F p n? 
25 or Less 38 2.91 0.54 
26 to 50 998 2.93 0.54 
51 to 75 6,070 3.06 0.52 
Greater than 75 3,874 3.36 0.50 
Total 10,980 3.15 0.54 F (8, 10,976) = 325.35 | <0.001 | 0.08 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 
Note: Effect size conventions for 7? are: 0.2 = small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without 
HR identifying information (n=35) are not included in the table. 
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Table F-4. Independent Sample t Test of Teachers Summative Ratings by Core Foundation 
Status, 2015-2016 


Std. 
Groups N Mean Deviation t p g 
Core Foundation 8,645 3.15 0.54 
Non-Core Foundation 2,335 3.19 0.53 
Total 10,980 3.15 0.54 t(1, 10,978) = 3.23 .001 0.07 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 
Note: Effect sizes are calculated for independent sample t-tests using Hedge’s g. Hedge’s g provides a measure of 
effect size weighted to account for different sample sizes. Effect size conventions for Hedge’s g are: 0.2 = 


small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without HR identifying information (n=35) are not 
included in the table. 


Table F-5. Independent Sample ¢ Test of Teachers Summative Ratings by Critical Shortage 
Status, 2015-2016 


Std. 
Groups N Mean Devishon t p g 
Critical Shortage 4,939 3.17 0.53 
Non-Critical Shortage 6,041 3.14 0.54 
Total 10,980 | 3.15 0.54 t(1, 10,978) =-2.68 | 0.007 | -0.06 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 
Note: Effect sizes are calculated for independent sample t-tests using Hedge’s g. Hedge’s g provides a measure of 
effect size weighted to account for different sample sizes. Effect size conventions for Hedge’s g are: 0.2 = 


small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without HR identifying information (n=35) are 
not included in the table. 


Table F-6. One-Way Between Analysis of Variance of Teachers’ Summative Rating by Teachers’ 


Total Years of Experience, 2015-2016 


Std. 
Groups N Mean Deviation F p n? 

New Teacher 1,173 2.79 0.53 

1 to 5 Years 3,425 3.12 0.52 

6 to 10 Years 2,116 3.23 0.52 

11 to 20 Years 2,616 3.24 0.52 

More than 20 Years 1,650 3.26 0.53 
Total 10,980 | 3.15 0.54 F (4, 10,975) = 189.52 | <0.001 | 0.06 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 


Note: Effect size conventions for 7? are: 0.2 = small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without 
HR identifying information (n=35) are not included in the table. 
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Table F-7. Independent Sample ¢ Test of Teachers Summative Ratings by Retention Status, 


2015-2016 
Std. 
Groups N Mean Deviation t p g 
Retained 9,511 3.19 0.51 
Exited 1,469 | 2.90 0.66 
Total 10,980 | 3.15 0.54 t(1, 10,978) =-16.15 | <0.001 | -0.54 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 

Note: Effect sizes are calculated for independent sample t-tests using Hedge’s g. Hedge’s g provides a measure of 
effect size weighted to account for different sample sizes. Effect size conventions for Hedge’s g are: 0.2 = 
small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without HR identifying information (n=35) are 
not included in the table. 


Table F-8. Independent Sample ¢ Test of Teachers Summative Ratings by Teacher Movement 


Status, 2015-2016 


Groups N Mean ie an t p g 
Remained at the same 8,763 | 3.21 0.50 
school 
Moved to a new location 748 3.03 0.55 
Total 9,511 3.19 0.51 t(1, 9,509) = 8.77 | <0.001 | 0.30 


Source: Teacher Appraisal and Development F&D Tool, 2015-2016 

Note: Effect sizes are calculated for independent sample t-tests using Hedge’s g. Hedge’s g provides a measure of 
effect size weighted to account for different sample sizes. Effect size conventions for Hedge’s g are: 0.2 = 
small effect, 0.5 = medium effect, 0.8 = large effect. Teachers without HR identifying information (n=35) are 
not included in the table. 
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